Developing Autonomous Navigation Algorithms Using Photorealistic Simulation

One of the principal problems in developing an automated driver is deciding how to handle the plethora of different decisions that have to be made in tactical situations. Insight on these problems can be obtained by the concurrent development of a driving simulator that allows both human and automated driving. A unique feature of the driving simulator that we have built is the ability to track eye movements within a freely moving VR helmet. This allows the assessment of exigencies in complex situations that can be used to guide the development of automated routines. INTRODUCTION Automated driving holds the promise of improving traffic safety, alleviating highway congestion and saving fuel. Currently there are increased research efforts in a number of countries directed towards developing such capabilities owing to recent advances that have made the computing necessary to support automated driving practical [1]. We are studying the problem of autonomous navigation in complex (urban) environments, with an emphasis on perception. We have built a driving simulator which generates photo-realistic images to be used as a testbed for the driving program. In this scenario, the program can control both the sensor (virtual camera) and the vehicle. The major difference between this system and previous simulators for driving programs is that in our case the actual images generated by the simulator are used as input for the perceptual routines in the driving program. Previous simulators have assumed that the vision problem can be handled and that some abThis research was supported by NIH/PHS research grant 1-P41-RR09283 stract description of the visual world is available to the agent. The simulator can also be used with human subjects who can drive the kart (depicted in figure 2) through the virtual environment while wearing head mounted displays (VR helmet). With an eye-tracker mounted inside the VR helmet, we can perform experiments on human attention during driving and obtain insights for designing the driving program. In particular the fixation point of the eyes at any moment is an indicator of the focus of attention for the human operator. Experiments show that this fixation point can be moved at the rate of one fixation every 0.3 to 1 second. Studying the motion of this fixation point provides information on how the human driver is allocating resources to solving the current set of tactical driving-related problems. THE SIMULATOR The simulator is designed for driving in an urban environment. The graphics engine is an SGI Onyx Infinite Reality computer, and the simulator was developed starting from the perfly demo program. The world simulator contains a description of the environment in which the vehicles navigate. This is represented as a database of roads, traffic signs, buildings, etc., with their 3D world coordinates (currently we are using the Performer Town database from SGI). The simulation is carried in discrete time steps. Figure 1 shows the two configurations in which the simulator can be used. In the first case (A), a human subject can sit in a kart (see figure 2) and drive through the virtual environment. There are controls for gas and brake (two pedals) and steering. The subject is also wearing a Virtual RealHead mounted display Polhemus Sensor Head position